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Recently researchers at the University of Illinois cou- 
pled a real pendulum to its virtual counterpart. They 
observed that the two pendulums suddenly start to move 
in synchrony if their lengths are sufficiently close [1 . In 
this synchronized state, the boundary between the real 
system and the virtual system is blurred, that is, the pen- 
dulums are in a mixed reality state. An instantaneous, 
bidirectional coupling is a prerequisite for mixed reality 
states. In the experimental setup, a motor moves the 
pivot point of the real pendulum slightly whenever the 
virtual pendulum moves, and vice versa. This requires a 
fast probe, a fast motor controller, and a fast computer 
that can run the virtual pendulum in real time and ac- 
complish the communication between the probe and the 
motor controller. 

When a real system is coupled to its virtual counter- 
part with a small, bidirectional, instantaneous coupling, 
the coupled system is an interreality system. A phase 
transition separates the two states of an interreality sys- 
tem: a dual reality state and a mixed reality state. 

In a mixed reality state, (i) the motion of the real sys- 
tem and the motion of the virtual system are in step and 
synchronized; (ii) the energy of the entire system is con- 
served; (iii) the amplitude of the motion increases to a 
large limiting amplitude; (iv) the real system strength- 
ens the motion of the virtual system and vice versa; and 
thus (v) the interreality system can generate a large out- 
put signal. 

In a dual reality state, (i) the motion of the real sys- 
tem and the motion of the virtual system are out of step 
and not synchronized; (ii) if the energy loss due to fric- 
tion is negligible, the energy of the real system and the 
virtual system are conserved separately; (iii) the ampli- 
tude of the motion decreases; (iv) the real system does 
not strengthen the motion of the virtual system and vice 
versa; and (v) the interreality system cannot generate a 
large output signal (see Figure [T]). 

Synchronization between two real systems has been 
studied extensively in the past [2 . For example, in a 
laser the dynamics of the electrons of different molecules 
can synchronize and thus produce a laser pulse of high in- 
tensity. Synchronization between real systems can be ex- 
ploited for desirable effects. Lasers are used in CD play- 
ers, laser pointers, and as energy efficient light sources. 
However, synchronization can also lead to destruction 
and catastrophes. At the opening of the Millennium 
Bridge in London to pedestrian traffic, the bridge started 
to sway sideways, which in turn caused the pedestrians 



to walk in step with the motion of bridge and the other 
pedestrians. Eventually the bridge started to sway at a 
dangerously large amplitude and had to be closed. The 
$32 million bridge reopened 2 years later, after $8.9 mil- 
lion worth of additional dampers were installed [3 . 

The significance of the recent work is the discovery 
that synchronization can occur between a real and a vir- 
tual system, and that the resulting mixed reality state is 
separated from the dual reality state by a phase transi- 
tion [1]. 
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FIG. 1: Properties of the motion of a system in a dual reality 
state and a mixed reality state. 

Mixed reality states can be used for control and for 
system identification, including the control of the human 
mind. Out-of-body experiences triggered by video feed- 
back are probably the best recent example of an observed 
mixed reality state. In the experiment, two video cam- 
eras are pointed at a subject from behind, and the sub- 
ject sees a video image of himself with 3D goggles 0[5]. 
The experimenter uses two sticks, one strokes the per- 
son's chest for 2 min, while the second stick moves just 
under the camera lenses, as if it were touching the vir- 
tual body. Then some of the subjects report an experi- 
ence where they appear to see themselves from outside 
their bodies. In this situation, the real system is the 
body of the subject, and the virtual system is the video 
image of the person. Since the video image is updated 
on a timescale which is much faster than the motion of 
the subject, and since the subject would recognize any 
change of the video image much faster than any motion 
on the body of the video image, there is instantaneous 
bidirectional interaction between the video image and the 
subject, that is, the subject and the video image are an 
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interreality system. Future versions of computer games 
such as Second Life may include a fast coupling to the 
real world with video cameras and probes, and may con- 
trol the real world with computer-controlled motors, ac- 
tuators, and switches. We would expect that eventually 
mixed reality states of virtual environments and the real 
world suddenly emerge. 

Because mixed reality states occur only when a virtual 
and a real system are sufficiently similar, we can use a 
virtual system to learn more about a real system. As we 
adjust the parameters of the virtual system to achieve 
mixed reality, we can develop estimates about the real 
system. An example is the Dynamic Neuron system. 
Typically one would measure the dynamics of a neuron 
and then systematically adjust the parameters of a model 
until a perfect match between the experimental dynamics 
and the experimental data is achieved. However, models 
of real neurons have numerous parameters, far too many 
for a systematic search and the experimental data are of- 
ten too noisy. To overcome this problem, Kullmann et al. 
coupled a real neuron to its virtual counterpart with an 
instantaneous interaction. When the real system and the 
virtual system synchronize, that is, enter into a mixed re- 
ality state, the model is assumed to be good and thereby 
certain parameters may be identified ^ 
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steering wheel and the other controls of the car to syn- 
chronize the motion of the car with its image. The wheels 
of car respond almost instantaneously to the turn of the 
steering wheel and the driver sees and feels the motion 
of the car quickly, hence the car and the driver are an 
interreality system. If the car does what the driver ex- 
pects it to do, this may be considered a mixed reality 
state. If the car behaves unexpectedly, then the two may 
be considered a dual reality state. Usually the onset of 
the unexpected motion is sudden: there is a phase tran- 
sition from a mixed reality state, where the car behaves 
as expected, to a dual reality state, where the driver has 
lost control over the car. 

Experienced drivers think ahead, that is, if they see 
difficult road conditions coming, they imagine themselves 
driving through the difficult area before they actually get 
to it [7]. When they perform this mental rehearsal in 
preparation of an upcoming challenge, they fast-forward 
in time the imagined interreality system of themselves 
and their car. Just thinking about the upcoming haz- 
ard may cause them to slow down or change lanes right 
away, indicating that the simulated time-traveling inter- 
reality system is coupled to the real interreality system. 
This coupling reduces the risk that the driver loses con- 
trol over his car, that is, the coupling between the time- 
traveling imagined interreality system and the real inter- 
reality system is used to stabilize the mixed reality state 
of the driver and his car. 

In the near future, computer hardware may become 
fast enough that it is not only possible to create mixed 
reality states, but to simulate the corresponding interre- 
ality system in real time or even have it time travel. The 
time-traveling simulated interreality system can be used 
to stabilize mixed reality states and make mixed reality 
states even more important. 
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FIG. 2: Synchronization in coupled real and virtual systems. 



When computers become even faster, it will be possible 
to couple an interreality system with its simulated coun- 
terpart. One of the unique advantages of the simulated 
interreality system is that it can fast-forward in time, or 
explore the past. In other words, the simulated inter- 
reality system can "time travel" (see Figure |2|. When 
an interreality system is coupled to its time - traveling 
simulated counterpart, mixed reality states in the "real" 
interreality system can be stabilized. For instance, if a 
good driver is driving his car along the interstate, the 
driver has an image of the car, its surroundings, and its 
expected dynamical behavior in his mind, and uses the 
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